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Abstract — In this paper, we introduce a new support recovery 
algorithm from noisy measurements called Bayesian hypothesis 
test via belief propagation (BHT-BP). BHT-BP focuses on sparse 
support recovery rather than sparse signal estimation. The key 
idea behind BHT-BP is to detect the support set of a sparse 
vector using hypothesis test where the posterior densities used in 
the test are obtained by aid of belief propagation (BP). Since BP 
provides precise posterior information using the noise statistic, 
BHT-BP can recover the support with robustness against the 
measurement noise. In addition, BHT-BP has low computational 
cost compared to the other algorithms by the use of BP. We show 
the support recovery performance of BHT-BP on the parameters 
(N, M, K, SNR) and compare the performance of BHT-BP to 
OMP and Lasso via numerical results. 

Index Terms — Sparsity pattern recovery, support recovery, 
Bayesian hypothesis test, compressed sensing, belief propagation, 
sparse matrix 

I. Introduction 

Support recovery (also known as sparsity pattern recovery) 
refers to the problem of finding the support set corresponding 
to K nonzero elements of a sparse vector x £ M. N from a noisy 
measurement vector z 6 M. M . This problem is fundamentally 
important to solve underdetermined systems (M < N) be- 
cause once the support set is known, the system is simplified 
to an overdetermined system, which can be solved by the 
conventional least square approach. Therefore, the support 
recovery is associated with a broad variety of underdetermined 
problems such as compressed sensing [1], subset selection in 
regression [2], sparse approximation Q. 

Recently, a plenty body of works has analyzed the perfor- 
mance of the support recovery Wainwright investigated 
the performance of the support recovery with Lasso in j4j, 
and Fletcher and Rangan analyzed that of OMP in (5) . In 
addition, the theoretical limit of the support recovery has been 
discussed in terms of maximum likelihood (ML) [6|,[7|, and 
information theory [8|,[9|. These theoretical works reveal that 
current practical algorithms have a potentially large gap from 
the theoretic limit; therefore, developing practical algorithms 
which approach the limit is an open challenge. 

One line of approaches is sparse recovery using sparse 
matrices [12|-[15|. These works are inspired by the success 

This work was proceeded in IEEE Statistical Signal Processing Workshop 
(SSP), Ann Arbor, Aug. 2012 



of low-density parity-check codes [11|,[10|. The use of the 
sparse matrix enables simple and fast measurement generation. 
In addition, these approaches can be made more attractive if 
they are applied in conjunction with belief propagation (BP). 
BP replaces the recovery process by iterative message-passing 
processes. This replacement reduces the computational cost to 
the 0(N log N) order E). 

Motivated by such previous works, in this paper, we propose 
a new support recovery algorithm using BP called Bayesian 
hypothesis test via BP (BHT-BP). BHT-BP utilizes a hypoth- 
esis test to detects the support set from noisy measurements, 
where the posterior density used in the test is provided by BP. 
Hence, BHT-BP has low computational cost from the use of 
BP and noise robustness from the hypothesis test. 

In our previous work lfT5ll . BHT-BP was a part to provide 
support set information for the process of sparse signal esti- 
mation. Differently from 1151 . this paper aims to investigate 
performance of support recovery by BHT-BP rather than the 
signal estimation. We show the support recovery performance 
of BHT-BP on the parameters (N, M, K, SNR) and demon- 
strate the superiority of BHT-BP compared to support recovery 
by OMP and Lasso via simulation results. 

II. Problem Formulation 

A. Signal Model 

Let x £ ~R N denote a random X-sparse vector with a 
state vector s(x) indicating the support set of x; therefore 
||s(x)|| = K. Each element Sj £ s(x) is defined as 

s^{l> for all i£{l,...,N}. (1) 

We limit our discussion to the random vector x whose ele- 
ments are i.i.d. random variables. Then, the decoder observes 
a measurement vector z £ M. M , given as 

z = *x + n, (2) 

where xo E R N is a deterministic realization of x; and 
n ~ jV(0, cr^Ijif) is an additive Gaussian noise vector. 
For the measurement matrix 4>, we employ sparse-Bernoulli 
matrices # £ {0, 1, -l} MxiV with rank(*) < M and 
M < N. Namely, in the matrix, sparsely nonzero elements 
are equiprobably equal to 1 or —1. In addition, we fix the 
column weight of <1> to L such that \\4>jth coi\\ 2 = L, 
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B. Problem Statement 

The goal of the decoder is to detect the support set s(xo) 
from z where each supportive state Si(x~o,i) is independently 
detected in each element unit, given as: 

Pr{ Si = 0|z} |o iforaU , i? } 
Prjsi = l\z\ Hl 

where Ho : Si(xo,i) = and Hi : Si(xo^) = 1 are two 
possible hypotheses. 

To see the performance of the algorithms, we measure the 
state error rate (SER) between the detected support s(xo) and 
the true support s(xo), given as 

#{i G {1, N}\ Sl (x ^) ^ Si(x ,i)} 



SER 



N 



(4) 



We are interested in the SER performance as a function of 
undersampling ratio M/N for a veriety of signal sparsity K 
and SNR defined as 

2 

(5) 



R||<|>X„ 

SNR : = 10 log! n „ J' 2 dB . 



^ Mai 
III. Proposed Algorithm 

A. Prior Model 

We first specify our prior model since the proposed al- 
gorithm is derived on the basis of the Bayesian rule. By 
associating state variable Si, we model the prior density of 
Xi using a spike- and- slab model originating in a two-state 
mixture density as follows: 

fx(x) := qf x (x\s = l) + (l-q)f x (x\s = 0) 

= qAf(x;Q,a 2 x ) + (l-q)6(x), (6) 

where S(x) indicates a Dirac distribution having nonzero value 
between x G [0— ,0+] and f S(x)dx = 1; q = J£ is the 
sparsity rate. In addition, we drop the index i from the prior 
density under the assumption of i.i.d. elements. The spike-and- 
slab prior has been widely employed in Bayesian inference 
problems fl6l . 

B. Hypothesis Detection of Support 

In order to perform the hypothesis test in d3l, the decoder 
needs to calculate a probability ratio p^r^rb} ■ By factorizing 
over Xi, the ratio becomes 

Pr{s, = 0|z} = /Pr{ 3i = Q\z,x l }f Xt {x\z)dx flo i 
Pr{s l = l|z} /Pr{s s : = l\z,Xi}f Xi (x\z)dx h x 

where f Xi (x\z) denotes the posterior density of Xi given 
z. In Pr{si\z,Xi} — Pr{si\xi} holds true since the 
measurements z do not provide any additional information on 
Si given x^. Using the Bayesian rule and the prior information, 
the test in |7) is rewritten as 



] -f Xi {x\z)dx m Pr{s = 0} 



(8) 



where 7 := r^qj ■ Therefore, the probability ratio is obtained 
from the corresponding posterior and prior densities. The 
overall flow of the hypothesis test for a supportive state 
detection is shown in Fig{T] 
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Fig. 1. Bayesian hypothesis test for a state detection 

C. Belief Propagation for Posterior Approximation 

The posterior density f x .(x\z) used in the hypothesis test 
is obtained by BR Using Bayesian rule, we can represent the 
posterior density f x .{x\z) in the form of Posterior = Prior x 
given as' 



f Xi (x\z) = f x (x) X 



fz(z\x t ) 

/.(*) ' 



(9) 



If the measurement matrix $ is sufficiently sparse such that the 
corresponding bipartite graph is tree-like, we postulate that the 
elements of z associated with Xi are independent each other 
ifTUll . Under the tree-like assumption, we can decompose the 
likelihood density f z (z\xi) to the product of densities: 



f x Jx\z) (X f x (x) x Y[ fzj{A x i)- 



(10) 



Since each element of z is represented by the sum of indepen- 
dent random variables with we can expressed f Zj (z\xi) as 
linear convolution of associated density functions, given as 



f Zj {z\xi) = 
6(z-Zj)<8 f nj {n) ® 



® /«»(«) 



(ID 



The essence of the BP-process is to obtain an approxima- 
tion of f Xi {x\%) from iterative mutual update of probability 
messages. The message update rule is formulated based on 
(fT0]),(fTT]i. We follow the rule introduced in 1151 . given as 



:= T) 



fx(x) X JJ 



(12) 



\®5{z-z j )®f n] (n) 1 (13) 



for all G {1,...,N} x {1,...,M} : \<pji\ = 1, where 

® and (^) are the operator for linear convolution and the 
linear convolution of a sequence of functions, respectively; 
rj(-) denote a normalization function for probability densities;, 
and I denotes the iteration index. The probability messages 
sn^,j and by_j,j are mutually updated via BP-iterations. Then, 



we approximate the posterior density f Xi (x\z) after a certain 
number of iterations /* as follows: 



f Xi (x\z) « n 



fx{x) 



n b 



(14) 



IV. Numerical Results 

We demonstrate the performance of BHT-BP via simulation 
results. We consider the SER performance given in Q and 
take 1000 Monte Carlo trials for each experimental point to 
show the average performance. In each trial, we generate a 
deterministic sparse vector xo with N = 128. The generation 
of Xi belonging to the support set follows zero mean Gaussian 
distribution with a x = 10; but we restrict the magnitude level 
to a x /5 < \xi\ < 3er x . For the measurement matrix <&, BHT- 
BP uses sparse-Bernoulli matrix with L = 3. 

Figj2] shows the SER performance as a function of under- 
sampling ratio M /N for a variety of SNR and K. Naturally, 
BHT-BP is well performed as SNR increases. However, BHT- 
BP works similarly beyond 30 dB as shown in Fig|2]-(c),(d). 
Regarding the signal sparsity K, BHT-BP needs at least 
SNR > 30 dB to recover the support with SER below the 
1/N « 0.0078 for K < 18 if M/N is sufficient. 

Fig(3] shows the advantages of BHT-BP compared to OMP 
lUTl and Lasso lfl8l . The source codes of OMP and Lasso 
were obtained from SparseLab 2.0 package (available at 
http://sparselab.stanford.edu/). OMP and Lasso use a Gaussian 
measurement matrix having the same column energy as the 
sparse-Bernoulli matrix for fairness, i.e., \\(/>j,Gaussian\\2 = 
II ^Sparse II 2 = L. In addition, we choose the if -largest values 
from xo for the support detection of OMP and Lasso. We 
generate Xo with N = 128 and K = 12 in this simulation. At 
SNR=10 dB, we can see that BHT-BP outperforms Lasso and 
OMP as shown in Fig[3]-(a), because the use of noise statis- 
tic /„. (n) in the BP-process provides the precise posterior 
information, and it leads to reducing of misdetection of the 
supportive state sb,i m the hypothesis test. Nevertheless, the 
SER curves at SNR=10 dB are far from SER = 1/N due 
to the noise effect. Such a fact is supported by the theoretical 
work in (6| where the ML decoder needs M/N > 1.02 for 
exact support recovery when SNR=10 dB. 

As SNR increases, all algorithms are gradually performed 
similarly, but Lasso and OMP have slightly lower cross point 
to SER= 1/N as shown in Fig(3}(b),(c),(d) and Table [i] 
Such a result is caused by sensing inefficiency from the use 
of sparse measurement matrices [8]. However, BHT-BP has 
advantage on low computational cost and the fast measurement 
generation with the sparse matrix. Indeed, BHT-BP has the 
lower cost 0(N log N) by aid of BP, than OMP O(KNM) 
and Lasso 0(NM 2 ). 
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Fig. 2. SER performance of BHT-BP over M/N for a variety of 
SNR and K: (a) SNR=10 dB, (b) SNR=20 dB, (c) SNR=30 dB, (d) 
SNR=50 dB. 
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Fig. 3. SER performance comparison to OMP and Lasso over M/N 
when K = 12: (a) SNR=10 dB, (b) SNR=20 dB, (c) SNR=30 dB, 
(d) SNR=50 dB. 



TABLE I 

Cross point in M/N to SER= 1/N when K = 12 



Algorithms / SNR 


50 dB 


30 dB 


20 dB 


10 dB 


BHT-BP 


0.357 


0.375 


0.575 




Lasso 


0.350 


0.361 


0.550 




OMP 


0.331 


0.335 


0.552 




ML limit for exact 
recovery in (6) 


0.101 


0.1108 


0.1935 


1.021 



V. Summary 

We proposed a new support recovery algorithm using belief 
propagation called BHT-BP. Our proposed algorithm utilizes 
hypothesis test to detects the support set from noisy measure- 
ments where posterior used in the test is provided by belief 
propagation. Our numerical results showed that the proposed 



algorithm outperforms OMP and Lasso in the low SNR 
regime, and becomes working similarly as SNR increases. 
However, the proposed algorithm still has strength in terms of 
low computational cost and the fast measurement generation 
by the use of the sparse matrix and belief propagation. 
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